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A  Partition  of  Small  Group  Predeciaion  Performance 
Into  Informational  and  Social  Components* 


The  differences  in  decision  making  abilities  between  individuals  and 
face-to-face,  interacting  groups  has  been  a  subject  of  long-standing 
interest.  The  purpose  of  our  study  is  to  present  and  demonstrate  new 
theoretical  and  methodological  techniques  for  partitioning  and  identifying 
the  sources  of  performance  differences  between  groups  and  individuals.  We 
illustrate  these  techniques  with  the  predecision  processes  of  hypothesis  and 
act  generation. 

Predeciaion  processes  are  the  cognitive  processes  that  precede  the 
selection  of  a  course  of  action.  Hypothesis  generation  is  a  predecision 
process  involving  the  generation  of  possible  explanations  (sometimes  called 
"states  of  the  world")  to  account  for  the  data  within  a  problem  context  (see 
Gettys  and  Fisher.  1979.)  Act  generation  is  a  predecision  process  which 
involves  generating  the  various  courses  of  action  which  are  available  to  the 
decision  maker. 

Previous  research  examining  hypothesis  generation  by  individuals  has 
demonstrated  deficiencies  in  every  context  examined  (Fisher,  Gettys,  Manning, 
Mehle,  &  Baca,  in  press;  Manning,  Gettys,  Nicewander,  Fisher.  &  Mehle,  1980; 
Mehle,  Gettys,  Manning,  Baca,  &  Fisher,  1981;  Gettys,  Mehle,  Baca,  Fisher,  & 
Manning.  Mote  2.)  Research  on  act  generation  by  individuals  has  yielded 
similar  conclusions  (Pita,  Sachs,  &  Heerbooth,  1980;  Gettys,  Manning,  & 
Casey.  Mote  3;  Pliske,  Gettys,  Manning,  &  Casey.  Mote  4).  In  realistic 
situations,  individuals  often  produce  sets  of  hypotheses  or  acts  which  are 
insufficient,  both  in  terms  of  quantity  and  quality,  to  allow  an  optimal  or 
nearly  optimal  choice  to  be  made. 

A  possible  remedy  for  such  deficiencies  would  be  to  replace  individuals 
with  face-to-face  groups  or  alternatively  with  nominal  groups  (Osborn,  1957). 
Members  of  a  face-to-face  group  can  pool  their  knowledge  and,  in  addition, 
the  responses  of  one  individual  in  a  face-to-face  group  may  serve  as 
synergistic  retrieval  cues  or  prompts  which  cause  others  to  retrieve 
additional  information.  In  a  nominal  group  (as  defined  by  Taylor,  Berry  and 
Block,  1958),  there  is  no  interaction  between  group  members.  Instead,  the 
contributions  of  individuals  who  worked  independently  (hereafter  referred  to 
as  "solo  individuals"  or  "solo  subjects")  are  pooled  post  hoc. 

Do  face-to-face  or  nominal  groups  significantly  outperform  solo 
individuals  in  hypothesis  or  act  generation  tasks?  Previous  studies  which  are 
pertinent  to  this  question  involve  experimental  tasks  which  share  an 
important  characteristic,  divergence,  with  the  tasks  of  interest  in  the 
present  study,  A  divergent  task  is  a  task  in  which  there  is  no  single 
"correct"  solution.  Rather,  there  are  many  or  an  infinite  number  of  solutions 
which  cannot  be  generated  by  any  straightforward  mechanical  rule.  Most 
studies  have  dealt  with  face-to-face  group,  nominal  group,  and  individual 
performance  in  divergent  tasks.  These  tasks  are  most  frequently  referred  to 
as  "brainstorming"  tasks,  although  terms  such  as  "creative",  "eureka",  "idea 
generation"  and  "imagination"  are  also  used. 


A  surprising  result  of  the  research  involving  divergent  tasks  is  that 
the  old  adage  "two  heads  are  better  than  one"  is  not  always  true.  For 
example,  Lorge,  Fox,  Davitz  and  Brenner  (1958)  surveyed  a  number  of  studies 
that  contrasted  group  and  individual  performance  and  concluded  that  groups 
often  do  no  better  than  the  best  individual  in  the  group.  Campbell  (1968) 
found  that  group  responses  were  inferior  to  those  of  a  single  subject. 
However,  other  studies  of  tasks  requiring  complex  judgments  (Gustafson, 
Shukla,  Delbecq  and  Walster,  1973;  Hall,  Mouton  and  Blake,  1967;  Klugman, 
1947)  have  found  groups  to  be  superior  to  individuals. 

With  a  few  exceptions  investigators  have  reported  a  rather  robust 
result:  when  combined  by  some  algorithm,  the  responses  of  individuals  working 
alone  is  superior  to  the  responses  of  groups  whose  members  were  permitted  to 
interact  freely  (Bouchard,  1972;  Dunnette.  Campbell  and  Jaastad,  1963; 
Gordon,  1923;  Graham,  1977;  Harari  and  Graham,  1975;  Maginn  and  Harris,  1980; 
Street,  1974;  Stroop,  1932;  Taylor  et  al.,  1958;  Van  de  Ven  and  Delbecq, 
1971,  1974;  Zajonc,  1962).  Yet  in  some  sense  the  old  adage  should  be  true; 
two  heads  should  contain  more  information  than  one.  However,  this  information 
may  not  always  translate  into  superior  group  performance,  possibly  due  to 
social  factors  that  are  operative  in  a  group  situation. 

The  major  difficulty  with  previous  research  examining  group  performance 
in  divergent  tasks  is  that  there  has  been  no  attempt  to  separate  the 
informational  factors  from  the  social  factors  which  are  present  to  varying 
degrees  in  f ace-to-f ace  and  nominal  groups.  That  is,  there  have  been  no 
theoretical  or  methodological  techniques  to  permit  the  determination  of  the 
factors  responsible  for  performance  differences  between  individuals  and 
different  types  of  groups.  Therefore,  the  present  investigation  presents  two 
alternative  models  which  can  be  used  to  isolate  these  performance 
differences. 

Experiment  1  was  our  initial  attempt  to  separate  empirically  the 
components  of  group  performance.  The  two  component  model  explored  in 
Experiment  1  may  be  sufficient  to  identify  the  source  of  performance 
differences  between  groups  and  individuals  to  a  level  of  detail  sufficient 
for  some  purposes.  Experiment  2  elaborates  the  model  of  group  performance  by 
isolating  an  additional  component  and  generalising  the  results  to  another 
predecision  task.  The  three  component  model  of  group  performance  presented 
in  Experiment  2  may  be  useful  for  some  theoretical  purposes. 

A  two-component  partition  of  groep  performance 

The  effect  of  pooling  information.  Suppose  that  two  solo  individuals 
work  on  a  divergent  problem  solving  or  decision  making  task.  If  the  relevant 
information  produced  by  one  solo  individual  is  later  combined  with  that  of 
the  other,  a  nominal  group  of  size  two  is  formed.  We  would  expect  this 
combined  store  to  be  superior  to  that  of  either  solo  individual.  In  this 
sense,  two  heads  sbonld  contain  more  information  than  one 2. 

If  there  is  no  overlap  or  duplication  between  the  two  sets  of 
information,  two  heads  should  be,  on  the  average,  twice  as  good  as  one. 
However,  in  realistic  tasks,  there  will  be  some  redundancy  between  two  solo 
individuals'  information  stores  as  a  result  of  common  knowledge  and 
experience.  Thus,  the  performance  of  a  nominal  group  of  size  two,  Pg.  can  be 
symbolized  as  in  Equation  1.  Where  Ia  and  1^  represent  the  performance  of  two 
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•olo  individual*,  a  and  b,  and  represents  the  redundancy  in  performance 

which  results  from  the  pooled  information  store. 


which  results  from  the  pooled  information  store. 

Ia  +  Ib  1a«b  "  1aub  "  *N 

Given  the  above  assumptions,  we  can  conclude  that  two  heads  will  be 
better  than  one.  but,  due  to  redundancy,  they  will  not  be  twice  as  good  as 
one.  Thus, 

(2)  I  <  P„  <  21, 

where  I  is  the  average  performance  of  a  solo  individual  and  Pp  is  the  actual 
performance  of  a  nominal  group  of  sise  two.  In  an  average  sense,  each  time 
the  information  produced  by  a  new  solo  individual  is  added  to  the  pool,  the 
amount  of  unique,  mon-rednndant  information  contributed  by  a  solo  individual 
becomes  smaller  (cf.,  Steiner,  1966). 

The  only  factor  that  distinguishes  a  nominal  group  from  a  solo 
individual  is  the  pooling  of  responses  according  to  some  algorithm  (done  post 
hoc  by  the  experimenter).  Since  there  is  no  communication  between  group 
members,  effects  due  to  social  interaction  cannot  be  operative  in  a  nominal 
group. 

If facts  due  to  social  interaction.  A  face-to-face  group  provides  the 
opportunity  for  social  interaction.  Social  interaction  gives  rise  to  several 
factors  which  may  cause  the  performance  of  a  face-to-face  group  to  be 
different  from  that  of  a  solo  individual  or  a  nominal  group.  Some  of  these 
factors  are  "purely  social"  factors  such  as  motivation,  level  of  arousal, 
social  comparison  processes,  tendency  to  polarize,  cohes iveness, 
interpersonal  attraction,  social  facilitation,  dominance,  etc.  Social 
interaction  may  also  give  rise  to  "social  informational"  factors  which  are 
often  thought  to  be  synergistic  in  nature.  Examples  of  social  informational 
factors  include  the  combination  of  partial  information  from  different  group 
members,  and  the  generation  of  new  or  improved  ideas  as  a  result  of  the 
sharing  of  ideas  between  group  members. 

A  two-component  partition  of  group  performance.  The  simple  version  of 
the  proposed  model  includes  two  major  components  which  define  the  performance 
of  a  face-to-face  group  as  follows:  1)  an  information  component,  which 
includes  the  effect  of  pooling  information,  and  2)  a  component  due  to  social 
interaction,  which  includes  both  "purely"  social  factors  and  social 
informational  factors.  The  model  proposes  that  the  pooling  of  information  in 
a  face-to-face  group  is  mediated  by  social  interaction,  which  causes  more  or 
less  information  to  be  pooled  than  would  occur  in  the  absence  of  social 
interaction.  This  way  of  partitioning  face-to-face  group  performance  is 
represented  in  Equation  3. 

(3)  Pftf  -  Pp  +  S. 

where  Prtj  is  the  performance  of  a  face-to-face  group  and  S  is  the  effect  of 
social  interaction  on  face-to-face  group  performance.  Pp  should  be  positive 
(disregarding  misinformation),  but  S  may  be  either  positive  or  negative.  In 
practice,  an  estimate  of  the  effect  of  social  interaction  on  face-to-face 
group  performance  can  be  obtained  by  subtracting  nominal  group  performance 
from  face-to-face  group  performance. 
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Fast  research  and  the  two-component  model.  The  studies  cited  above  which 
have  shown  that  noninal  groups  are  superior  to  face-to-face  groups  are 
examples  in  which  social  interaction  has  a  negative  effect  on  the  pooling  of 
information.  On  the  other  hand,  Cohen,  Whitmyre  and  Funk  (1960)  found  that, 
in  one  experimental  condition,  face-to-face  dyads  were  superior  to  nominal 
dyads  in  a  brainstorming  task.  In  terms  of  our  partition  of  group 
performance,  this  is  an  example  in  which  social  interaction  caused  increased 
pooling  of  information.  The  manipulation  that  presumably  created  this 
increase  was  the  formation  of  cohesive  dyads. 

Jablin,  Seibold  and  Sorenson  (1977)  found  no  difference  between  nominal 
and  face-to-face  groups  in  brainstorming  performance.  In  terms  of  our 
partition,  this  finding  could  be  attributed  to  a  negligible  effect  of  social 
factors  on  the  pooling  of  information  in  the  face-to-face  groups.  However, 
the  power  of  their  experiment  was  too  low  to  permit  a  definite  conclusion. 

The  two-component  model  presented  above  makes  explicit  some  of  the  ideas 
that  have  been  more  or  less  implicit  in  previous  group  research.  He  propose 
that  the  performance  of  face-to-face  groups  will  or  will  not  exceed  the 
performance  of  nominal  groups  depending  on  how  social  interaction  affects  the 
pooling  of  information.  This  model  should  help  clarify  some  of  the 
conflicting  findings  reported  in  the  group  performance  literature  regarding 
the  potential  superiority  of  nominal  groups. 

In  addition  to  illustrating  how  the  model  may  be  used  to  partition  group 
performance.  Experiment  1  addresses  another  difficulty  with  past  research  on 
divergent  tasks.  This  difficulty  concerns  the  lack  of  agreement  among 
researchers  on  the  appropriate  dependent  measure  to  use  in  divergent  thinking 
tasks  (cf.,  Stein,  1975).  Much  of  the  early  work  on  divergent  thinking  used 
the  qmamtity  of  responses  as  the  primary  dependent  variable.  This  reflects 
the  assumption  made  by  brainstorming  proponents  that  the  quality  or 
usefulness  of  a  set  of  responses  is  linearly  related  to  the  quantity  of 
responses.  Although  counting  the  number  of  ideas  generated  by  a  subjects 
sounds  quite  easy,  there  is  often  some  ambiguity  in  identifying  the  number  of 
different  or  unique  ideas  a  subject  generated.  Some  researchers  (e.g., 
Dunnette  et  al.,  1963;  Taylor  et  al.,  1958)  have  tried  to  assess  the  quality 
of  the  ideas  generated  by  having  independent  raters  evaluate  the  various 
ideas. 


Unfortunately,  observed  differences  in  the  performance  of  nominal  and 
face-to-face  groups  seem  to  depend  to  some  extent  on  the  performance  measure 
used.  For  example,  Stein  (1975)  concludes 

.  .  .  when  both  nominal  and  real  groups  use  brainstorming  the 
former  outperform  the  latter  in  number  and  even  quality  of  ideas. 

But,  when  quality  is  studied  as  a  function  of  number  of  ideas  then 
differences  wash  out.  (p.  95) 

Experiment  1  avoids  the  problems  inherent  in  previous  measures  of  divergent 
thinking  performance  by  using  veridical  posterior  probabilities  as  a  measure 
of  quality  of  responses  in  a  hypothesis  generation  task.  This  measure  is 
explained  in  more  detail  below. 
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The  hypothesis  generstion  task. 


A  divergent  task,  as  we  have  shown,  asy  be  partitioned  into 
informational  and  social  components.  Hypothesis  generation  is  an  example  of  a 
divergent  task  because  the  object  of  the  task  is  to  generate  as  many 
explanations  as  possible  for  a  given  set  of  data.  These  explanations  should 
aeet  a  minimum  criterion  for  plausibility;  that  is,  they  should  be  consistent 
with  the  data  of  the  problem  and  be  likely  enough  to  be  useful  as  potential 
explanations  for  the  data. 

The  hypothesis  generation  task  used  in  this  study  is  somewhat  novel,  and 
for  that  reason  needs  some  justification  and  explanation.  Subjects  were  given 
a  course  that  a  student  at  the  University  of  Oklahoma  had  taken,  and  were 
instructed  to  list  as  many  plausible  majors  for  this  student  as  possible.  It 
was  possible  to  interrogate  the  master  files  of  student  enrollments  to 
determine  the  veridical  probabilistic  relationships  between  courses  and 
majors.  This  task  allows  a  more  sophisticated  qualitative  comparison  between 
nominal  and  face-to-face  groups  than  has  been  possible  using  other  tasks. 

The  'Ssajors  from  classes"  task  affords  three  advantages  over  traditional 
brainstorming  tasks:  1)  objective  determination  of  whether  two  responses  are 
equivalent,  2)  objective  evaluation  of  the  quality  of  responses,  and  3)  a 
single  measure  that  reflects  both  the  quantity  and  quality  of  an  individual's 
responses.  The  criterion  of  performance  was  objective  because  of  the 
availability  of  veridical  posterior  probabilities.  Thus,  it  was  possible  to 
objectively  compare  the  quality  of  responses  obtained  from  nominal  and  face- 
to-face  groups  by  adding  together  the  posterior  probabilities  of  each  of  the 
plausible  hypotheses  generated. 


Method 


Subjects 

The  subjects  were  80  female  students  enrolled  in  an  introductory 
psychology  course  at  the  University  of  Oklahoma.  Sixteen  of  the  subjects 
worked  individually;  the  remaining  64  worked  in  groups  of  four. 

Procedure 

The  stimuli  were  eight  written  problems  to  which  the  subjects  made 
written  responses.  For  each  problem,  subjects  were  given  a  brief  description 
of  a  course  offered  by  the  University  of  Oklahoma  and  the  associated  course 
number.  Courses  having  large  enrollments  were  chosen  for  use  as  problems. 
See  Mehle  et  al.  (1981)  for  further  discussion  of  this  paradigm. 

The  principle  manipulation  was  a  comparison  of  subjects  working  alone  to 
subjects  working  in  groups  of  four.  In  the  solo  condition,  each  of  16 
subjects  was  given  the  set  of  eight  problems.  These  subjects  were  instructed 
not  to  discuss  the  problems  with  each  other.  In  the  face-to-face  condition, 
16  groups  of  four  subjects  esch  were  run.  The  members  of  each  group  were 
instructed  to  work  together  to  generate  hypotheses  for  each  problem.  One 
group  member  was  chosen  to  mske  a  written  list  of  all  responses  made  by  the 
group.  The  individuals  and  groups  were  given  the  same  problems. 
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For  each  problem,  subjects  or  groups  were  given  five  minutes  to  list  all 
majors  of  an  "unknown  student"  who  took  the  given  course.  To  discourage 
subjects  from  making  a  "memory  dump”  of  all  possible  majors,  we  defined  a 
"plausible  major"  as  any  major  that  includes  at  least  two  percent  of  the 
students  who  had  taken  that  course.  The  concept  of  plausibility  is  desirable 
in  this  context  because  it  prevented  subjects  from  making  the  specious 
argument  that  any  major  is  possible  and  should,  therefore,  be  a  valid 
response.  (In  fact,  not  all  majors  take  all  courses.)  Therefore,  subjects 
were  instructed  to  include  a  major  only  if  two  percent  or  more  of  the 
students  who  took  the  course  had  thst  major.  Subjects  were  also  told  that 
they  would  not  be  penalized  if  they  happened  to  give  a  response  that  did  not 
meet  this  criterion. 


Results  and  Discussion 

The  dependent  measure  for  this  study  was  the  probability  that  the  major 
of  a  student  who  has  taken  the  given  course  was  contained  in  the  set  of 
hypothesis  generated  by  an  individual  or  group.  In  order  to  obtain  this 
measure  it  was  it  was  necessary  to  calculate  the  veridical  or  actual 
probability  of  any  set  of  hypotheses.  The  computer  master  record  of  the 
course  work  taken  by  undergraduate  students  at  the  University  of  Oklahoma  was 
analyzed.  For  these  analyses,  the  "population"  was  defined  as  the  current 
student  population  at  the  University  of  Oklahoma.  The  computer  file 
contained  information  on  the  courses  taken  over  the  previous  four  years  by 
students  currently  enrolled  at  the  university.  The  probabilistic 
relationships  between  classes  and  majors  obtained  by  analyzing  the  computer 
file  were  regarded  as  population  parameters  and  were  used  as  veridical 
probabilities.  An  analysis  of  this  type  was  done  for  each  of  the  classes 
presented  as  data  to  subjects.  The  task  permitted  calculation  of  the 
veridical  probability  that  the  true  state  of  the  world  would  be  included  in  a 
particular  set  of  majors. 

The  total  hypothesis  set  probability  was  calculated,  for  each  of  the  eight 
problems,  for  each  solo  subject  and  for  each  face-to-face  group,  by  adding 
together  the  posterior  probabilities  of  each  of  the  plausible  hypotheses.  The 
corresponding  expression  is: 

(4)  P(Correct)*  £P(M|C)  >  .02, 

where  P(Correct)  is  the  probability  that  the  set  of  hypotheses  contains  the 
correct  hypothesis  and  P(M|C)  is  the  probability  of  a  major  given  a 
particular  class.  In  accordance  with  the  instructions  given  to  subjects, 
majors  with  veridical  probabilities  of  less  than  .02  for  the  course  being 
considered  were  not  included. 

Equation  4  can  also  be  applied  in  evaluating  the  performance  of  a 
nominal  group.  In  this  case,  the  hypothesis  sets  of  the  individuals  who 
comprise  the  nominal  group  are  pooled  to  create  a  single  set  of  hypotheses. 
Redundant  hypotheses  are  deleted  from  this  list  and  the  performance  of  the 
nominal  group  is  determined  by  adding  the  posterior  probabilities  of  the 
remaining  hypotheses  according  to  Equation  4. 

‘s  an  ex*  'pie,  the  datum  for  one  problem  was:  "Sociology  1113. 
let.  *  —.ti'  co  Sociology."  One  solo  individual  generated  the  hypothesis 


set:  Art,  Music,  Mathematics,  Psychology.  Sociology.  History.  Education, 
Drama  and  Nursing.  The  veridical  probability  that  a  student  who  had  taken 
Sociology  1113  had  a  major  that  was  an  element  of  this  set  of  hypotheses  was 
calculated  to  be  22.1  percent.  Continuing  with  this  example,  a  face-to-face 
group  of  four  subjects  generated  the  set:  Sociology,  Psychology,  Social 
Work,  Chemistry,  Zoology,  Political  Science,  Nursing,  Journalism,  Economics, 
Drama,  Education  and  Engineering.  The  veridical  probability  of  this  set  was 
calculated  to  be  34.5  percent. 

Problem  means  for  the  solo  individuals,  face-to-face,  and  nominal  groups 
are  presented  in  Table  1.  The  column  entries  in  the  first  two  columns  are 
means  over  16  individuals  and  16  groups,  respectively.  The  third  column 
contains  the  means  of  the  1,820  possible  distinct  synthetic  groups  of  size 
four  (all  possible  combinations  of  16  taken  four  at  a  time.)  This  method  of 
calculating  nominal  group  performance  requires  a  considerable  amount  of 
computer  time.  However,  it  provides  a  more  accurate  estimate  of  nominal  group 
performance  using  the  available  data,  since  error  variability  due  to  the 
particular  random  assignment  of  solo  subjects  to  nominal  groups  is 
eliminated. 


Table  1 

Comparison  of  Individual,  Group  and  Nominal 
Hypothesis  Set  Probabilities 


Mean  Probability  of  hypothesis  sets  (in  percents) 


Problem 

Solo 

individuals 

Face-to-face 
groups  of  four 

Nominal 
groups  of 

A 

80.3 

85.9 

89.2 

B 

30.0 

39.6 

49.7 

C 

32.2 

44.1 

52.7 

D 

25.1 

34.5 

50.8 

E 

27.6 

40.9 

49.5 

F 

31.0 

42.6 

52.6 

G 

24.9 

30.0 

50.0 

16.7 


24.2 


37.6 


Examining  the  problem  means,  face-to-face  groups  were  superior  to  solo 
individuals  for  all  eight  problems.  Also,  nominal  groups  were  superior  to 
face-to-face  groups  and  to  solo  individuals  on  all  eight  problems.  The 
binomial  probability  of  one  condition  exceeding  another  by  chance  on  all 
eight  problems  is  less  than  .004  (i.e.,  0.5®  <  .004).  Therefore,  face-to-face 
groups  were  significantly  superior  to  solo  subjects  and  nominal  groups  were 
significantly  superior  to  face-to-face  groups. 

Estimating  the  model  components 

The  performance  partition  described  in  the  introduction  permits  the 
estimation  of  social  and  information  components  of  group  performance.  The 
information  component  estimates  the  increased  information  possessed  by  the 
group  due  to  the  pooling  of  their  information  stores.  This  component  is 
estimated  to  be  20.5  percent  from  the  data  collected  in  Experiment  1.  This 
was  computed  by  subtracting  the  average  performance  of  the  solo  individuals 
(33.5)  from  the  average  performance  of  nominal  groups  of  size  four  (54.0). 

The  social  interaction  factor  was  estimated  to  be  -11.3  percent  by 
subtracting  the  average  performance  of  nominal  groups  of  size  four  (54.0) 
from  the  average  performance  of  face-to-face  groups  of  size  four  (42.7).  In 
this  task,  social  interaction  was  found  to  impair  performance.  That  is, 
actual  social  (face-to-face)  interaction  resulted  in  poorer  performance  than 
the  post  hoc  pooling  of  solo  individual  performance  (i.e.,  nominal  groups). 
Because  face-to-face  groups  (42.7)  were  found  to  be  somewhat  superior  to  solo 
individuals  (33.5),  these  data  indicate  that  a  group  of  hypothesis  generators 
(both  nominal  and  face-to-face)  is  preferable  to  a  single  individual. 
However,  the  negative  social  interaction  component  estimated  above  indicates 
that  it  is  preferable  co  have  hypothesis  generators  work  alone  and  pool  their 
ideas  post  hoc,  rather  than  have  them  work  in  an  interacting  face-to-face 
group. 

As  the  estimates  given  above  illustrate,  partitioning  performance  into 
social  and  informational  components,  gives  new  insights  into  group  research 
issues.  Had  we  used  a  task  with  a  smaller  informational  component,  face-face 
groups  may  have  been  inferior  to  individuals.  Had  social  interaction 
facilitated  performance,  the  extent  of  this  facilitation  would  be  unknown 
unless  the  information  component  had  been  estimated.  This  technique  provides 
researchers  with  the  necessary  tool  to  isolate  information  and  social 
components,  and  thus  to  gain  a  greater  understanding  of  the  data. 
Furthermore,  using  a  dependent  variable  that  reflects  both  the  quality  and 
quantity  of  performance  leads  to  a  less  ambiguous  conclusion. 


EXPERIMENT  2 

Experiment  1  increased  the  generality  of  the  result  that  nominal  groups 
are  superior  to  face-to-face  groups  in  divergent  tasks  by  extending  it  to  a 
hypothesis  generation  task.  It  also  suggests  that  this  result  and  those  of 
others  were  obtained  because  social  interaction  actually  impairs  performance 
more  than  is  gained  by  pooling  and  sharing  information.  The  paradox  of  the 
superiority  of  nominal  groups  in  predecision  tasks  raises  an  interesting 
question.  Decision  makers  usvally  prefer  to  work  in  groups,  so  they  must 
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believe  tbat  working  in  a  group  is  more  productive  than  working  alone.  Do 
decision  makers  prefer  to  work  in  groups  for  social,  interpersonal  or  other 
reasons  (e.g.,  diffusion  cf  responsibility)  unrelated  to  the  decision  making 
performance  of  the  group?  Alternatively,  do  decision  makers  prefer  working  in 
groups  because  of  informational  benefits  of  group  participation  which  enhance 
the  decision  making  performance  of  the  group? 

Previous  studies  comparing  nominal  and  face-to-face  groups  have  not 
addressed  this  question.  To  our  knowledge,  no  studies  of  this  kind  (including 
Experiment  1  of  the  present  study)  have  partitioned  the  two  types  of  effects 
due  to  social  interaction  that  were  discussed  earlier.  One  of  these  effects 
involves  purely  social  factors  (e.g.,  motivation,  social  facilitation,  etc.) 
and  the  other  involves  social  informational  factors  which  are  often  thought 
to  be  synergistic  in  nature.  Both  types  of  effects  seem  intuitively  to  be 
crucial  determinants  of  predecision  performance. 

One  particular  social  informational  factor  is  especially  interesting 
because  it  is  one  of  the  major  aspects  of  group  behavior  upon  which  the 
brainstorming  paradigm  is  meant  to  capitalize.  This  phenomenon  has,  perhaps 
unfortunately,  been  labeled  "piggybacking"  (Day,  1980)  and  "hitch-hiking" 
(Stein,  1975);  it  involves  group  members  use  of  each  other's  ideas  as  aids 
(i.e.,  retrieval  cues  or  prompts)  in  generating  additional  or  improved  ideas. 
Investigators  have,  for  many  years,  recognized  that  hearing  the  ideas  of 
others  may  facilitate  recall  of  more  and  better  ideas.  For  example,  Osborn 
(1957)  suggested  that  "the  average  person  can  think  up  twice  as  many  ideas 
when  working  with  a  group  than  when  working  alone"  (pp.  228-229).  However, 
this  phenomenon  has  yet  to  be  demonstrated  experimentally. 

Dunnette  et  al.  (1963)  tried  to  optimize  brainstorming  performance  by 
having  subjects  work  on  a  brainstorming  problem  in  face-to-face  groups  and 
then  solo.  Using  the  same  assignment  of  subjects  to  groups,  nominal  groups 
were  formed  from  the  sets  of  solo  responses.  These  nominal  groups 
outperformed  other  nominal  groups  made  up  of  individuals  who  worked  on  the 
problem  without  the  benefit  of  prior  group  interaction.  While  this  result  is 
quite  interesting  in  terms  of  improving  brainstorming  performance,  it  does 
not  allow  unequivocal  interpretation  of  the  reason  for  the  improvement.  The 
improvement  was  not  necessarily  due  to  any  sort  of  "piggybacking."  Many 
alternative  explanations  exist,  such  as  time  spent  in  the  task  and  rehearsal 
and  recall  factors. 

A  method  is  needed  which  will  allow  unequivocal  identification  of 
potentially  beneficial  or  detrimental  informational  effects  of  group 
participation  over  and  above  those  explained  by  the  simple  pooling  of 
information  or  by  purely  social  aspects  of  group  interaction.  Therefore,  the 
three-factor  partition  of  group  performance  presented  in  the  next  section 
isolates  an  additional  social  component  which  estimates  the  effect  of  social 
interaction  on  memory  retrieval  and  problem  solving  ability. 

A  three-factor  partition  of  group  performance 

The  model  presented  earlier  (see  Equation  3)  separated  face-to-face 
group  performance  into  two  components.  One  component,  Pg,  is  due  to  pooling 
of  information  and  the  other,  S,  is  due  to  social  interaction.  We  now  wish  to 
factor  the  social  interaction  component  into  two  meaningful,  independent 
components  by  including  a  social  informational  factor.  The  resulting  three 
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component  partition  of  face-to-face  group  performance  is  represented  in 
Equation  S. 

pftf  "  PH  ♦  sie  +  sni» 

where  Pftf  and  Pg  are  defined  as  in  Equation  3.  Sj»  represents  the  effect  on 
face-to-xace  group  performance  of  the  additional  information  that  becomes 
accessible  (or  perhaps  inaccessible)  due  to  the  exchange  of  information 
between  members  of  the  group.  He  will  call  this  the  "information  exchange" 
component.  Sn£  represents  the  effect  of  purely  social,  non-inf ormational 
factors  on  face-to-face  group  performance.  Both  Sfe  and  SQf  may  be  either 
positive  or  negative,  depending  on  whether  they  improve  or  impair 
performance. 


The  magnitude  of  the  information  exchange  component  can  be  estimated 
empirically  by  comparing  nominal  group  performance  with  the  performance  of  an 
information  exchange  group  whose  members  exchange  information  in  the  absence 
of  social  interaction,  Pja*  Thus,  the  information  exchange  component  can  be 
evaluated  as  in  Equation  6. 


sie  "  pie  "  PN 

if  pie  exceeds  Pg,  then  the  information  exchange  component  is  facilitory.  If 
Pg  exceeds  p£e>  tnen  the  information  exchange  component  is  inhibitory. 

The  magnitude  of  the  purely  social,  non-informat ional  component  can  be 
estimated  empirically  by  comparing  face-to-face  group  performance  with 
information  exchange  group  performance.  This  component  can  be  evaluated  sb  in 
Equation  7. 
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If  Pf£f  exceeds  P£e,  then  the  non-inf ormational,  social  component  is 
facilitory.  If  P^e  exceeds  Pftf,  then  this  component  is  inhibitory. 

A  tecfamiqwe  for  estimating  the  information  exchange  component 


In  order  to  demonstrate  the  effect  of  the  information  exchange 
component,  an  experimental  method  is  needed  which  will  allow  measurement  of 
the  potential  informational  advantages  (or  disadvantages)  of  group 
interaction  without  contamination  by  purely  social  factors.  Two  general 
strategies  exist  for  estimating  the  information  exchange  component.  One 
strategy  involves  using  face-to-face  groups,  but  attempting  to  modify  the 
interaction  of  group  members  in  order  to  minimise  inhibitory  social  pressure. 
The  other  strategy  involves  using  a  group  structure  which  is  basically 
nominal  in  nature,  but  which  allows  for  exchange  of  information  between  group 
members. 


The  brainstorming  paradigm  described  by  Osborn  (1957)  is  a  good  example 
of  the  strategy  involving  face-to-face  groups.  Osborn  attempted  to  minimise 
social  pressure  by  prohibiting  criticism  and  giving  instructions  such  as  "the 
more  ideas  the  better,  the  wilder  the  ideas  the  better."  While  such 
instructions  are  undoubtedly  a  potent  manipulation,  they  would  not  accomplish 
our  purpose  of  completely  eliminating  the  social  factors  associated  with 
face-to-face  groups.  Dunnette  et  al.  (1963)  conclude  that: 
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In  spite  of  the  stimulus  of  group  brainstorming  and  our  specific 
directive  to  avoid  all  criticism,  it  was  apparent  that  these 
persons  [research  scientists  and  advertising  personnel,  most 
of  whom  were  veil  acquainted  with  each  other]  were  inhibited 
simply  by  the  presence  of  other  group  members.  The  central  idea 
underlying  brainstorming  of  placing  a  moratorium  on  all  criticism 
is  a  good  one.  It  appears,  however,  that  group  participation 
still  contains  certain  inhibitory  influences  which  are  not  easily 
dissipated,  (p.  37) 

Although  Street  (1974)  has  shown  that  the  mere  presence  of  other  group 
members  has  an  insignificant  effect  on  brainstorming  performance,  it  seems 
any  sort  of  face-to-face  interaction  is  bound  to  entail  social  pressure  of 
some  kind. 

For  the  present  study  we  chose  to  estimate  the  information  exchange 
component  in  the  absence  of  face-to-face  interaction.  We  developed  a 
technique  in  which  subjects  worked  on  an  act  generation  problem  in  physical 
isolation  from  other  subjects  and  were  led  to  believe  they  were  being  aided 
by  a  computer  when,  in  fact,  they  were  exchanging  ideas  with  another  subject. 
The  difference  in  performance  between  groups  whose  members  exchanged  acts 
(which  we  call  the  information  exchange  condition)  and  traditional  nominal 
groups  provides  an  estimate  of  the  information  exchange  component  (see 
Equation  6).  If  this  component  is  found  to  be  positive,  it  can  be  concluded 
that  there  ia  a  benefit  of  group  interaction  which  is  due  neither  to  the 
information  available  to  group  members  independently  (in  the  absence  of 
interaction),  nor  to  purely  social  effects  of  group  interaction. 

If  the  information  exchange  component  is  found  to  be  negative,  it  can  be 
concluded  that  there  is  a  detrimental  effect  of  group  interaction  which  can 
be  explained  neither  by  group  members'  independent  lack  of  knowledge  nor  by 
purely  social  factors.  A  negative  information  exchange  component  could,  for 
example,  be  due  to  subjects  causing  each  other  to  "get  in  a  rut"  and  pursue  a 
single  train  of  thought  to  the  exclusion  of  other  good  ideas.  Alternatively, 
the  "getting  in  a  rut"  phenomenon  could  be  created  by  purely  social  factors. 
In  this  case,  the  phenomenon  would  occur  in  face-to-face  groups,  but  not  in 
information  exchange  groups.  Therefore,  referring  to  Equation  5.  the 
phenomenon  would  decrease  Sftf,  but  have  no  effect  on  S^e. 

In  terms  of  our  extended  partition  of  gioup  performance  (Equation  6)  the 
previously-mentioned  finding  of  Cohen  et  al.  (1960)  that  cohesive  face-face 
groups  were  superior  to  nominal  groups  may  be  attributable  to  either  (or 
both)  the  information  exchange  component  or  the  social,  non-inf ormational 
component.  The  only  conclusion  that  can  be  drawn  from  their  data  is  that  the 
sum  of  these  components  is  positive  and,  therefore,  there  is  an  overall 
facilitory  effect  of  group  interaction.  It  cannot  be  concluded  that  both 
social  components  were  positive. 

Experiment  2  illustrates  how  our  extended  partition  of  group  performance 
can  be  used  to  identify  the  source(s)  of  performance  differences  between 
different  kinds  groups.  Nominal,  face-to-face,  and  information  exchange 
groups  were  compared  in  terms  of  quality  of  sets  of  responses  produced.  A  new 
experimental  technique  is  presented  which  allows  exchange  of  information 
between  group  members  without  contamination  by  the  purely  social  factors  and 


social  demand  characteristics  that  are  generally  active  in  face-to-face 
groups. 

Experiment  2  generalizes  the  partitioning  technique  to  a  different 
predecision  task,  act  generation.  The  hypothesis  generation  task  used  in 
Experiment  1  permitted  the  use  of  veridical  probabilities  as  a  measure  of 
performance.  However,  for  many  real  world  decision  problems  veridical 
probabilities  are  not  available.  Experiment  2  also  illustrates  how  the 
partitioning  technique  can  be  used  with  another  dependent  measure.  Subjects 
were  given  a  realistic  decision  problem  and  were  asked  to  generate  actions 
which  could  be  taken  to  solve  the  problem.  Their  performance  was  score*,  in 
terms  of  quality,  as  well  as  quantity,  of  the  actions  they  generated. 


Method 


Subjects 

The  subjects  were  139  male  and  female  introductory  psychology  students 
from  the  University  of  Oklahoma  who  participated  in  the  experiment  for  course 
credit.  Subjects  with  typing  skills  were  solicited.  Those  who  failed  a  20 
word  per  minute  typing  test  were  not  permitted  to  participate  in  the 
experiment.  Subjects  were  given  three  chances  to  pass  the  test.  Data  from 
three  subjects  in  the  solo  condition,  three  pairs  of  subjects  in  the 
information  exchange  condition  and  five  pairs  of  subjects  in  the  face-to-face 
condition  were  not  analyzed  because  their  post-experimental  questionnaires 
indicated  they  ran  out  of  time  before  completing  the  experiment.  The  data 
reported  below  are  based  on  40  subjects  per  condition. 

Problem 


The  decision  problem  we  used  was  called  the  Parking  problem.  This 
problem  asked  subjects  to  imagine  themselves  to  be  members  of  a  student 
committee  which  is  supposed  to  come  up  with  alternative  solutions  to  the 
parking  problem  at  the  University  of  Oklahoma.  Previous  research  (Gettys  et 
al.,  Note  3)  has  indicated  that  college  subjects  find  this  act  generation 
task  realistic  and  meaningful.  It  is  also  an  attractive  problem,  because  it 
is  a  specific  example  of  a  more  general  type  of  problem  faced  by  many 
decision-makers.  That  is,  the  Parking  problem  is  essentially  a  shortage 
problem  in  which  the  decision-maker  is  faced  with  a  shortage  of  a  valuable 
commodity  (i.e.,  parking  places.) 

The  use  of  a  covariate  to  reduce  error  variance 

We  included  a  pretest  as  a  covariate  in  the  present  experiment  in 
anticipation  of  large  individual  differences  in  act  generation  performance  as 
have  been  found  by  Gettys  et  al.  (Note  3)  and  Pliske  et  al.  (Note  4).  Manning 
et  al.  (1980)  found  that  the  Alternate  Uses  creativity  test  developed  by 
Christensen,  Guilford,  Merrifield  and  Wilson  (I960)  and  discussed  by 
Guilford,  Christensen,  Merrifield  and  Wilson  (1978)  is  a  good  predictor  of 
hypothesis  generation  performance.  Since  hypothesis  and  act  generation  are 
similar  cognitive  tasks,  the  Alternate  Uses  test  may  also  be  predictive  of 
act  generation  performance. 
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Our  version  of  the  Alternate  Uses  test  is  a  paper  and  pencil  test  which 
includes  10  items.  For  each  item,  the  subject  is  to  list  all  possible  uses 
for  an  object  other  than  the  use  commonly  associated  with  the  object.  For 
example  a  coat  hanger  may  be  used  unlock  a  car  or  roast  food  over  a  fire;  a 
brick  may  be  used  as  a  doorstop  or  to  crack  walnuts,  etc. 

Procedure 

The  Alternate  Uses  test  was  administered  during  the  first  15  minutes  of 
the  session.  Subjects  were  then  randomly  assigned  to  conditions.  For  the 
remainder  of  the  experiment,  subjects  were  seated  in  front  of  a  CRT  terminal 
which  was  controlled  by  a  computer.  They  interacted  with  the  computer  by 
reading  textual  material  displayed  on  the  CRT  screen  and  by  typing  in  their 
responses  via  the  keyboard. 

Experimental  conditiona 

The  three  experimental  conditions  will  be  referred  to  as  solo, 
information  exchange,  and  face-to-face.  In  the  solo  condition,  a  single 
subject  was  seated  before  the  terminal  and  no  other  subjects  were  in  the 
room.  In  the  face-to-face  condition,  two  subjects  were  seated  before  the 
terminal.  Prior  to  the  experimental  session,  one  of  these  subjects  was 
randomly  designated  as  the  typist.  (However,  if  this  subject  failed  the 
typing  test,  the  other  subject  was  allowed  to  attempt  it.) 

In  the  information  exchange  condition,  two  subjects  participated 
simultaneously  in  different  rooms  using  different  computers.  Subjects  were 
unaware  of  each  other's  presence.  As  in  the  solo  condition,  a  single  subject 
was  seated  in  front  of  the  CRT.  In  the  information  exchange  condition,  the 
lower  half  of  the  CRT  screen  was  dedicated  to  the  act  the  subject  was 
entering  and  the  upper  half  to  acts  "from  the  computer."  Acts  typed  in  by  one 
subject  were  automatically  sent  to  the  other  subject. 

Instructions 

Subjects  were  instructed  to  generate  all  possible  acts  that  might  solve 
the  problem.  However,  they  were  cautioned  not  to  suggest  "frivolous  or 
counterproductive"  acts  or  minor  variations  of  a  single  act.  They  were  told 
that  the  experiment  would  consist  of  only  two  problems,  a  practice  problem 
and  an  experimental  problem  and  that  there  would  be  no  time  limit  on  the 
experimental  problem.  In  addition  to  reading  these  and  related  instructions 
on  the  screen,  subjects  were  given  a  paper  copy  of  the  instructions  in  case 
they  wished  to  refer  back  to  it  later  in  the  experiment.  In  the  face-to-face 
condition,  aubjects  were  told  that  they  would  be  working  together  as  a  team 
and  that  the  performance  of  their  team  was  the  variable  of  interest  to  the 
experimenter . 

Information  exchange  subjects  were  presented  with  the  following 
additional  instructions  (both  on  the  screen  and  on  paper): 

The  computer  has  been  programmed  to  attempt  to  help  you  come  up  with 
actions  which  could  be  taken.  Stored  in  the  computer's  electronic 
memory  are  a  large  number  of  possible  actions  for  the  problems  you 
will  be  working  on.  These  actions  were  suggested  by  subjects  in  a 
previous  experiment.  You  will  not  be  required  to  read  through  all  of 
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Che  action*  Chat  the  subjects  suggested*  as  that  night  take  several 
hours.  Instead*  the  computer  program  will  do  this  for  you.  Each  time 
you  finish  typing  in  an  action  that  you  have  thought  of,  the  program 
will  attempt  to  interpret  the  basic  ideas  involved  in  your 
suggestion.  Then,  while  you  are  thinking  of  more  actions  and  typing 
them  in*  the  program  will  automatically  search  through  the 
computer's  memory  for  actions  that  are  related  to  yours.  Whenever 
such  an  action  is  found*  it  will  be  displayed  on  the  screen  and  you 
will  hear  a  "beep."  Also,  whenever  this  happens*  the  action  will  be 
added  to  your  list  of  actions  (and  to  your  score  in  the  experiment) 
just  a*  if  you  had  thought  of  it  yourself. 

The  computer  program  is  still  in  the  developmental  stages  and  has 
only  a  limited  ability  to  interpret  actions  that  you  suggest. 
Therefore,  don't  be  alarmed  if  actions  appear  on  the  screen  which 
are  not  closely  related  to  your  suggestions.  Also*  some  of  the 
actions  that  appear  may  be  quite  bad.  No  attempt  has  been  made  to 
"weed  out"  ridiculous  or  counterproductive  actions  from  the 
computer's  memory.  However,  such  actions  will  not  affect  your  score. 

When  an  action  from  the  computer  appears  on  the  screen*  be  sure  to 
read  it  right  away.  Otherwise  it  may  be  replaced  by  something  new 
before  you  have  finished  reading  it. 


The  practice  problem 

Subjects  in  all  groups  were  given  a  practice  problem  in  which  they  were 
asked  to  think  of  acts  which  could  be  taken  if  one  ran  out  of  gas  on  the 
highway  and  had  no  money.  They  entered  one  act  at  a  time.  Each  time  subjects 
indicated  that  an  act  was  complete,  they  were  required  to  explain  how  the  act 
might  solve  the  problem.  This  was  done  to  aid  the  experimenters  in 
determining  what  subjects  hoped  to  accomplish  with  the  acts  they  suggested. 
When  the  explanation  was  finished*  the  act  and  explanation  disappeared  from 
the  screen  and  subjects  were  given  the  opportunity  to  enter  another  act. 

For  the  solo  and  face-to-face  conditions'  practice  problems*  subjects 
were  required  to  enter  and  explain  three  acts.  When  the  third  act  and 
explanation  was  complete  the  computer  displayed  three  exemplar  acts  generated 
by  a  creative  subject  to  encourage  them  to  think  of  as  many  acts  as  possible. 

For  the  information  exchange  condition*  the  computers  were 
interconnected  and  programmed  such  that  until  both  subjects  had  entered  a 
minimum  of  three  acts  in  response  to  the  practice  problem*  both  subjects  were 
required  to  continue  entering  additional  acts.  This  was  necessary  to  insure 
that  both  subjects  started  the  experimental  problem  at  the  same  time. 
Contrary  to  the  instructions,  the  computers  had  no  ability  to  interpret  the 
substance  of  subjects'  acts.  After  subjects  completed  their  first  act  (and 
explanation)  one  of  the  three  exemplar  acts  was  displayed  on  the  screen.  The 
other  two  exemplar  acts  were  displayed  at  approximately  two  minute  intervals 
while  the  subject  was  entering  other  acts.  Only  one  exemplar  act  was  visible 
at  a  time.  When  an  act  from  the  computer  appeared  on  the  screen,  the  keyboard 
was  disabled  for  one  or  two  seconds,  after  which  the  subject  could  continue 
typing. 
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In  all  conditions,  subjects  were  not  told  bow  many  acts  to  enter  for  the 
practice  problem.  They  were  told  that  the  computer  would  proceed 
automatically  at  the  proper  tiae.  The  experimenter  helped  them  during  the 
practice  problea  to  insure  that  they  understood  the  instructions  and  were 
able  to  interact  with  the  computer  without  difficulty. 

The  ezperiaeatal  problem 

After  subjects  were  shown  the  exemplar  acts,  the  text  describing  the 
parking  problem  on  the  University  of  Oklahoma's  caapus  was  presented  on  the 
screen.  The  text  of  the  problem  is  given  in  Gettys  et  al.  (Note  3).  Subjects 
were  also  given  a  paper  copy  of  this  text  for  later  reference.  They  were 
reainded  to  enter  all  reasonable  acts  which  might  have  even  a  reaote 
possibility  of  solving  the  problem.  For  the  experimental  problea,  any  nuaber 
of  acts  could  be  entered  and  no  exaaple  acts  were  given  to  solo  or  face-face 
subjects.  Otherwise,  the  instructions  for  each  condition  were  the  same  as  for 
the  practice  problea  and  subjects  interacted  with  the  computer  in  the  same 
way. 


Each  tiae  an  information  exchange  subject  finished  entering  an  act  for 
the  experiaental  problea,  it  was  transmitted  iaaediately  to  the  other 
computer  and  it  appeared  on  the  upper  half  of  the  other  subject's  screen.  In 
this  way,  the  two  subjects  exchanged  acts  throughout  the  experiaental 
session,  without  being  aware  that  they  were  doing  so. 

All  subjects  were  verbally  instructed  to  inf  ora  the  experiaenter  "if  you 
completely  run  out  of  reasonable  ideas."  When  they  inforaed  the  experiaenter 
that  they  had  indeed  run  out  of  ideas,  they  were  asked  to  fill  out  a  post- 
experiaental  questionnaire.  They  were  not  told  about  the  questionnaire  prior 
to  this  tiae.  The  questionnaire  included  open-ended,  Multiple  choice,  and 
Likert  scale  iteas  concerning  strategies  subjects  used  to  come  up  with  acts, 
how  many  aore  acts  there  aight  be,  possible  reasons  for  terminating  the 
experiment  (e.g.,  did  you  run  out  of  time?  Did  you  find  yourself  thinking  of 
aore  and  aore  bad  ideas?)  and  whether  the  computer  (information  exchange 
condition)  or  their  partner  (face-to-face  condition)  was  a  help  or  a 
hindrance.  All  subjects  worked  independently  on  the  questionnaire. 


lesults  and  Discussion 


Amber  of  acts  generated 

The  Alternate  Uses  test  was  significantly  correlated  with  each  of  the 
dependent  variables  examined.  Correlations  ranged  fron  .26  to  .37.  Therefore, 
the  Alternate  Uses  scores  were  included  as  a  covariate  in  all  analyses  of 
variance  and  are  reflected  in  all  ¥  statistics  reported  below. 

In  the  face-to-face  condition,  a  single  set  of  act  generation  data  was 
collected  for  each  pair  of  subjects.  The  average  nuaber  of  acts  generated  by 
face-to-face  pairs  was  11.3.  For  the  solo  and  information  exchange 
conditions,  a  set  of  data  was  collected  froa  each  individual.  However,  the 
unit  of  primary  interest  for  all  conditions  is  the  combined  performance  of 
two  subjects.  The  mean  nuaber  of  acts  generated  bv  information  exchange  pairs 
was  27.  For  the  solo  condition,  noainal  groups  (alternatively  called  "solo 
pairs"  or  Nominal  pairs")  were  formed  by  randomly  pairing  subjects.  The  aean 


number  of  acts  generated  by  nominal  groups  was  19.1.  There  was  a  significant 
difference  in  mean  number  of  acts  generated,  F( 2, 56)a16.81, MSe"65.7, 
p<.0001.  Individual  comparisons  show  that  all  three  of  these  means  are 
significantly  different  from  each  other  (p<.05). 

The  mean  number  of  acts  generated  by  the  40  solo  subjects  (not  pairs) 
was  9.6.  Since  this  mean  was  lower  than  all  three  meana  given  above  for 
pairs,  it  is  of  interest  to  compare  the  number  of  acts  generated  by  solo 
subjects  with  that  for  the  lowest  of  the  others,  face-to-face  pairs.  Twenty 
solo  subjects  were  selected  at  random  for  comparison  with  face-to-face  pairs. 
The  mean  number  of  acts  generated  by  these  twenty  subjects  was  9.9.  This  mean 
was  not  significantly  different  from  the  mean  number  of  acts  generated  by 
face-to-face  pairs.  Thus,  two  people  working  as  a  face-to-face  team  did  not 
generate  significantly  more  acts  than  a  single  person  working  alone. 

Classification  of  acts 

Many  of  the  acts  generated  by  the  subjects  were  conceptually  quite 
similar.  For  example,  one  subject  might  suggest  that  the  University  build  a 
parking  lot  next  to  the  library,  whereas  another  subject  might  suggest 
building  a  parking  lot  next  to  the  student  union.  He  classified  the  1148  acts 
generated  by  the  subjects  into  a  40  category  hierarchical  decision  tree  to 
determine  how  many  unique  acts  subjects  had  generated.  This  tree  was 
developed  by  Pliske,  et  al.  (Mote  4)  (where  it  is  presented  in  its  entirety) 
in  order  to  capture  the  major  ideas  expressed  in  subjects'  acts  for  solving 
the  Parking  problem.  The  two  acts  described  above  would  both  be  classified 
into  a  category  labeled  "Build  new  parking  on  University  land." 

The  tree  was  intended  to  include  the  broadest  possible  range  of 
reasonable  actions  for  solving  the  problem.  This  was  done  because  of  the 
emphasis  that  brainstorming  advocates  place  on  generating  a  wide  variety  of 
different  ideas.  It  is  assumed  that  refining  or  "tweaking"  (Edwards,  Mote  5) 
in  order  to  arrive  at  the  best  possible  variation  of  a  general  solution  would 
not  be  done  until  all  of  the  relevant  ideas  have  been  considered. 

Two  raters  independently  classified  the  acts  into  the  tree.  The  raters 
were  not  told  which  condition  the  acts  were  from.  The  percentage  of  agreement 
for  the  two  raters  was  92  percent.  The  acts  that  the  raters  disagreed  on  were 
discussed  in  conjunction  with  several  of  the  authors  until  everyone  agreed  on 
an  appropriate  classification. 

lesults  based  on  number  of  unique  acts 

About  4  percent  of  the  acts  had  to  be  eliminated  because  they  were 
either  blatantly  counterproductive  (e.g.  "nuke  the  university")  or  because 
they  were  incoherent  and  impossible  to  interpret.  Forty  reasonable,  unique 
acts  were  possible  under  our  classification  system.  After  the  "bad"  acts  were 
eliminated,  solo  pairs  generated  an  average  of  12.25  unique,  reasonable  acts. 
Information  exchange  pairs  generated  an  average  of  14.1  and  face-to-face 
pairs  generated  an  average  of  8.3  unique,  reasonable  acts.  Analysis  of 
variance  on  these  three  means  was  significant,  F(2,56)<*16.69,MScaa9.29, 
p<.0001.  Individual  comparisons  revealed  that  all  three  pairwise  combinations 
of  means  were  significantly  different  from  each  other  (p<.05).  The  mean 
number  of  unique,  reasonable  acts  generated  by  the  20  randomly  chosen  solo 
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subjects  (sot  pairs)  was  7.3.  This  mean  was  not  significantly  different  from 
that  for  face-to-face  pairs. 

Quantity  of  unique  responses  has  been  the  traditional  dependent  variable 
in  brainstorming  research.  Our  conclusions  about  performance  based  only  on 
the  quantity  of  unique  responses  can  be  summarised  as  follows: 

face-to-face  pairs  <  solo  pairs  <  information  exchange  pairs 

(or  solo  subjects) 

A  single  person  working  alone  produces  as  many  unique,  reasonable  solutions 
as  two  people  working  together  face-to-face.  However,  two  heads  are  better 
than  one,  in  an  average  sense,  because  solo  pairs  perform  better  than 
individual  solo  subjects.  Furthermore,  there  is  a  benefit  of  the  exchange  of 
ideas  between  subjects,  since  information  exchange  pairs  performed  better 
than  would  be  predicted  from  the  simple  pooling  of  the  responses  produced  by 
an  equal  number  of  individuals  working  solo. 

Utility  estimation  procedure 

In  addition  to  developing  the  act  classification  tree,  Pliske,  et  al. 
(Mote  4)  obtained  utility  estimates  for  each  of  the  40  categories  included  in 
the  tree.  The  utility  estimates  were  global  estimates  of  the  quality  of 
actions.  They  took  into  account  the  cost  of  implementing  an  action  and  the 
likelihoods  of  the  various  outcomes  which  may  result  from  the  action.  Pliske, 
et  al.  elicited  utility  estimates  from  five  "experts"  (university 
administrators)  who  were  knowledgeable  about  the  parking  problem  on  campus. 
The  utility  estimation  procedure  involved  four  steps  beginning  with 
categorizing  40  generic  actions,  which  corresponded  to  the  40  categories  in 
the  act  classification  tree,  into  five  categories  according  to  usefulness  and 
ending  when  all  acts  had  been  rated  on  a  0  to  100  point  utility  scale.  The 
correlations  between  the  experts'  utility  ratings  were  fairly  high.  A  utility 
value  was  assigned  to  each  of  the  40  categories  by  finding  the  median  of  the 
experts'  ratings  for  that  generic  act. 

Keeulta  based  on  quality  of  acts 

Using  our  classification  system  and  associated  utility  values,  it  was 
possible  to  characterize  the  quality  of  the  acts  suggested  by  summing  the 
utilities  of  the  unique  acts.  (The  "bad"  acts  were  assigned  the  value  0.)  The 
mean  cumulative  utility  for  solo  pairs  was  637,  for  information  exchange 
pairs  681,  and  for  face-to-face  pairs  401.  An  analysis  of  variance  on  these 
three  means  was  significant.  F(2,56)"19.61,MSt“19659.69,  p<.0001.  Individual 
comparisons  showed  that  the  face-to-face  condition  was  inferior  to  both  of 
the  other  conditions  (p<.05),  but  solo  pairs  and  information  exchange  pairs 
did  not  differ  significantly.  The  mean  cumulative  utility  for  the  20  randomly 
chosen  solo  subjects  (not  pairs)  was  339.  This  mean  was  not  significantly 
different  from  that  for  the  face-to-face  condition. 

Therefore,  in  terms  ol  quality  of  unique  acts,  both  a  nominal  group 
(solo  pairs)  and  an  information  exchange  group  were  found  to  be  superior  to  a 
face-to-face  group.  However,  there  was  not  a  significant  improvement  in  the 
quality  of  ideas  generated  by  the  information  exchange  pairs  as  compared  to 
the  solo  pairs.  Furthermore,  a  face-to-face  group  of  two  people  was  not  found 
to  be,  on  the  average,  any  better  than  a  single  person  working  alone. 
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A  similar  result  is  obtained  when  the  quality  of  the  beat  act  that  was 
generated  is  examined.  This  measure  is  of  particular  interest  in  tasks  that 
involve  mutually  exclusive  acts,  such  that  only  one  of  the  possible  acts  can 
be  implemented.  For  the  Parking  problem,  some  of  the  acts  preclude  each  other 
and  some  do  not.  However,  the  utility  of  the  best  act  generated  by  a  subject 
or  group  is  of  interest  since  it  is  an  act  that  would  surely  be  implemented. 

The  average  utility  of  the  highest  utility  act  generated  by  solo  pairs 
was  90.6,  for  information  exchange  pairs  89.2,  and  for  face-to-f ace  pairs 
81.15.  An  analysis  of  variance  on  these  means  was  significant,  F( 2, 56) *8.7 7, 
MSe"895.0,  p<.001.  Individual  comparisons  showed  that  the  face-to-f  ace 
condition  was  inferior  to  both  of  the  other  conditions  (p<.05),  but  the  other 
two  conditions  did  not  differ  significantly.  The  average  utility  of  the 
highest  utility  act  generated  by  the  20  randomly  chosen  solo  subjects  (not 
pairs)  was  85.05.  This  mean  was  not  significantly  different  from  that  for  the 
face-to-face  condition. 

The  results  presented  above  indicate  that  observed  differences  between 
nominal,  information  exchange,  and  face-to-face  groups  depend  to  some  extent 
on  the  performance  measure  used.  It  appears  that  the  preferred  dependent 
measure  would  be  one  which  reflected  both  the  quantity  and  quality  of  a 
subject's  performance.  The  cumulative  utility  measure  presented  above 
encompasses  both  aspects  of  a  subject's  performance  because  it  is  affected  by 
both  the  number  and  quality  of  the  acts  generated  by  a  subject.  Therefore,  we 
illustrate  the  three  component  model  presented  in  the  introduction  to 
Experiment  2  using  the  cumulative  utility  of  the  acts  generated  by  the 
subjects. 

Intimating  the  model  components. 

We  now  have  all  of  the  information  necessary  to  estimate  the  magnitude 
of  the  information  exchange  component  and  the  purely  social,  non- 
informationsl  component  of  face-to-face  group  performance.  From  Equation  6. 
the  information  exchange  component,  S^c  ia  equal  to  information  exchange 
group  performance  minus  nominal  group  performance.  Thus, 

Sie  -  681  -  637  -  44 

The  information  exchange  component  results  in  an  increase  in  performance  of 
44  utiles.  This  represents  a  6.5  percent  increase  in  performance. 

From  Equation  5.  the  magnitude  of  the  "purely"  social,  non-informat ional 
component,  SD£,  is  the  difference  between  face-to-face  group  performance  and 
information  exchange  group  performance.  Thus, 

sni  "  pftf  ~  pie  "  401  "  681  "  "280 

The  purely  social  component  results  in  a  decrement  in  face-to-face  group 
performance  of  280  utiles.  This  represents  a  41  percent  decrement  in 
performance  due  to  purely  social,  non-inform at ional  factors. 
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It  is  also  possible  to  estimate  the  total  effect  of  social  interaction, 
S  (see  Equation  3).  This  effect  is  equal  to  the  difference  between  face-face 
group  performance  and  nominal  group  performance.  (Alternatively,  it  is  the 
sum  of  8^e  and  Sn£.)  Thus. 

\ 

8  “  pftf  -  P||  •  401  -  637  -  -23  6 

social  interaction  results  in  a  decrement  in  face-to-face  group  performance 
of  236  utiles.  This  represents  a  37  percent  decrement  in  performance. 

It  can  be  seen  that  the  overall  effect  of  social  interaction  is  actually 
a  combination  of  two  factors.  When  the  information  exchange  component  was  not 
partitioned  out  of  face-to-face  group  performance,  social  interaction 
appeared  to  have  a  smaller  inhibitory  effect  on  performance.  In  fact,  the 
effect  of  purely  social  factors  on  face-to-face  group  performance  is  so  large 
that  it  more  than  negates  the  informational  benefits  that  also  result  from 
social  interaction. 

The  redundancy  of  information  contributed  by  iadiv ideal s 

As  discussed  earlier,  if  there  is  no  overlap  between  the  information 
held  by  two  solo  individuals  regarding  a  task,  then  a  nominal  group  formed 
from  the  responses  of  the  two  individuals  should,  on  the  average,  perform 
twice  as  well  as  the  average  solo  individual.  Any  redundancy  between  the  two 
individuals'  information  stores  would  result  in  nominal  group  performance 
lass  than  twica  as  good  as  the  average  solo  individual. 

A  measure  of  the  redundancy  of  information  contained  by  solo  individuals 
in  the  present  experiment  can  be  obtained  by  comparing  the  performance  of  a 
nominal  group  of  sise  two  with  two  times  the  performance  of  a  solo 
individual.  In  terms  of  cumulative  utility,  the  average  performance  of 
nominal  groups  was  637  utiles  while  twice  the  average  performance  of  solo 
subjects  was  718.  Therefore,  considering  the  effect  of  information  on  act 
generation  performance,  on  the  average,  about  40.3  utiles  or  11  percent  of 
the  information  possessed  by  one  nominal  group  member  was  redundant  with  that 
of  the  other  member. 

Differences  between  information  exchange  and  nominal  groopa 

Information  exchange  pairs  generated  an  average  of  41  percent  more 
unclassified  acts  than  nominal  pairs.  However,  the  average  number  of  unique 
acts  (the  number  of  acts  after  classification  into  the  40  category  tree)  for 
information  exchange  pairs  was  only  13  percent  greater  than  that  for  nominal 
pairs.  This  result  indicates  that  information  exchange  pairs  generated 
multiple  acts  in  the  same  category  more  frequently  than  did  nominal  pairs.  A 
question  arises  as  to  the  nature  of  these  variations  of  the  generic  acts. 
Possibly,  information  exchange  subjects  generated  minor  improvements  on  each 
other's  (or  their  own)  acts.  In  fact,  this  is  what  one  might  expect  a  subject 
to  do  if  the  information  "exchanged"  helped  the  subjects  to  retrieve 
additional  information  with  which  they  could  create  new  solutions  to  the 
problem.  Por  this  "tweaking"  behavior  to  be  most  productive,  they  should  have 
generated  variations  of  the  best  acts,  since  these  are  the  ones  most  likely 
to  be  implemented. 
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The  dependent  measures  of  quality  which  we  have  considered  thus  far  do 
not  address  this  question.  As  discussed  above,  information  exchange  pairs 
performed  only  6.9  percent  better  than  the  nominal  pairs  in  terms  of 
cumulative  utility.  Also,  nominal  pairs  performed  slightly  better,  although 
not  significantly,  in  terms  of  the  average  utility  of  the  highest  utility  act 
generated.  These  measures  are  affected  only  by  whether  a  category  was 
generated.  They  are  not  sensitive  to  multiple  acts  classified  in  the  same 
category. 

A  new  dependent  measure  of  quality,  the  average  utility  per  act,  was 
computed  for  each  subject.  This  measure  was  based  on  the  utility  of  all  the 
acts  generated  (i.e.,  unclassified  acts)  without  regard  to  uniqueness. 
Although  there  were  some  redundant  acts  in  all  conditions,  if  information 
exchange  pairs  had  a  greater  tendency  to  generate  redundant  acts  of  high 
utility,  their  average  utility  scores  (when  those  scores  are  based  on  the 
utility  of  all  acts  generated)  should  be  higher  than  the  average  utility 
scores  for  the  other  conditions.  However,  the  data  from  the  present  study 
indicate  that  these  means  were  nearly  the  same  for  all  conditions  and  for 
solo  subjects.  The  average  utility  per  act  for  the  information  exchange  pairs 
was  46.1;  the  average  utility  per  act  for  the  solo  pairs  was  46.0;  the 
average  utility  per  act  for  the-face-to  face  pairs  was  46.0;  and  for  the  20 
randomly  chosen  solo  individuals  the  average  utility  per  act  was  46.5. 

He  can  conclude  from  this  analysis  that  although  information  exchange 
subjects  generated  more  variations  of  each  of  the  40  generic  acts  which 
comprised  the  40  category  decision  tree  used  to  classify  subjects'  responses, 
these  variations  were  not  restricted  to  the  high-utility  generic  acts.  That 
is,  the  average  utility  or  quality  of  the  acts  generated  by  subjects  in  the 
information  exchange  condition  did  not  differ  from  the  average  utility  of  the 
acts  generated  by  the  other  conditions.  The  information  exchange  component 
of  group  performance  in  the  act  generation  task  used  in  the  present 
experiment  appears  to  primarily  affect  the  quantity,  but  not  the  quality,  of 
the  acts  produced. 

One  possible  explanation  for  this  finding  is  related  to  the  fact  that 
subjects  were  instructed  to  generate  all  possible,  reasonable  acts  regardless 
of  utility.  They  may  indeed  have  given  little  consideration  to  utility  in 
deciding  which  ideas  to  pursue  in  attempting  to  generate  additional  acts.  The 
information  exchange  component  would  probably  be  larger  in  tasks  that  are 
less  divergent  in  nature.  Zn  such  tasks,  the  "piggybacking"  phenomenon  of 
improving  on  each  others  ideas  might  be  more  more  facilitory. 

Differences  between  nominal  and  face-to-face  groups 

The  performance  of  face-to-face  pairs  was  found  to  be  inferior  to  both 
the  solo  pairs  and  the  information  exchange  pairs  un  all  of  the  dependent 
measures  examined  in  Experiment  2.  This  is  consistent  with  many  previous 
studies  which  compared  nominal  and  real  group  performance  on  divergent 
thinking  tasks  (cf.,  Stein,  1975).  The  face-to-face  pairs  in  Experiment  2 
performed  no  better  than  a  solo  individual.  This  result  is  somewhat 
surprising  given  the  increased  amount  of  information  which  should  become 
available  due  to  the  informational  and  the  information  exchange  components 
of  group  performance.  Why  does  the  social,  non-informat ional  component  have 
such  a  large,  negative  effect  on  act  generation  performance? 
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The  present  study  was  not  designed  to  explain  why  the  social, 
non informational  component  often  has  a  negative  effect  on  group  performance. 
However,  data  collected  in  post -experimental  questionnaires  rule  out  several 
potential  explanations.  When  two  people  work  in  a  face-to-face  group,  they 
might  tend  to  evaluate  the  actions  which  come  to  mind  before  verbalizing 
these  actions  to  the  other  group  member.  Subjects  might  be  more  critical  of 
their  own  act  generation  behavior  when  they  are  placed  in  a  social  situation. 
Subjects  in  the  present  study  were  asked  to  indicate  on  a  10-point  scale  how 
often  they  thought  of  actions  but  then  rejected  them  as  being  implausible  or 
unrealistic  solutions.  The  0  point  on  the  scale  was  labeled  "never"  and  the 
10  point  was  labeled  "always."  The  mean  response  for  all  three  conditions  was 
3.0.  This  does  not  support  the  hypothesis  that  subjects  in  the  face-to-face 
condition  are  more  likely  to  evaluate  and  reject  the  actions  they  think  of 
than  subjects  in  the  other  conditions. 

Subjects  were  also  asked  to  give  us  their  opinions  on  several  statements 
describing  the  experiment.  These  included  statements  like  the  following: 
'^his  experiment  was  challenging",  "This  experiment  was  fun,"  etc.  Subjects 
responded  on  a  10-point  scale  where  the  0  point  was  labeled  "disagree 
strongly"  and  the  10  point  was  labeled  "agree  strongly.”  The  only  statistical 
difference  of  interest  was  found  for  a  statement  given  to  subjects  in  the 
information  exchange  and  face-to-face  conditions  which  stated  "The  computer 
[or  my  partner]  helped  me  think  of  new  ideas.”  The  mean  response  for  this 
statement  from  subjects  in  the  information  exchange  condition  was  6.3  and  the 
mean  response  from  subjects  in  the  face-to-face  condition  was  7.7, 
F(l,78)*7.64,  MSe«5.3,  p<.01.  This  result  suggests  that  although  subjects' 
act  generation  performance  in  face-to-face  pairs  was  inferior  to  information 
exchange  pairs,  subjects  in  the  face-to-face  condition  rated  their  partner  as 
being  more  helpful  than  subjects  in  the  information  exchange  condition  rated 
the  computer.  This  result  seems  to  reflect  the  common  assumption  made  by 
people  that  working  with  others  in  a  face-to-face,  interacting  group  will 
improve  their  performance. 


General  Discussion 

In  Experiment  1  a  two-component  model  was  used  to  estimate  the  magnitude 
of  the  informational  and  social  components  of  face-to-face  group  performance 
in  a  hypothesis  generation  task.  Four  heads  were  found  to  be  superior  to  one 
in  terms  of  quality  performance.  However,  a  large  difference  was  found 
depending  on  whether  the  four  people  worked  together  or  separately.  Social 
interaction  resulted  in  considerable  impairment  of  hypothesis  generation 
performance. 

In  Experiment  2.  a  three-component  model  was  used  to  estimate  the 
magnitude  of  the  informational,  information  exchange  and  purely  social 
components  of  face-to-face  group  performance  in  an  act  generation  task.  Two 
heads  were  found  to  be  better  than  one  only  when  there  was  no  social 
interaction  between  the  two  people.  Thus,  the  overall  effect  of  social 
interaction  was  negative.  However,  separation  of  the  effect  of  social 
interaction  into  information  exchange  and  purely  social  components  revealed  a 
small  performance  benefit  produced  by  group  interaction.  This  benefit  was  due 
to  the  exchange  of  information  between  group  members.  The  information 
exchange  effect  has  been  impossible  to  assess  in  other  studies,  because 
exchange  of  information  between  group  members  is  generally  confounded  with 


the  accompanying  social  factors  which  tend  to  have  a  powerful  deleterious 
effect  on  group  performance. 

The  three-component  model  of  face-to-face  group  performance  allows  the 
experimenter  to  identify  more  precisely  the  source(s)  of  performance 
differences  between  various  types  of  groups  in  divergent  tasks.  However,  a 
drawback  of  the  three-component  model  as  opposed  to  the  two-component  model 
is  that  50  percent  more  subjects  are  needed.  The  two-component  model  may 
suffice  in  many  situations. 

He  see  possibilities  for  elaborating  the  three-component  model  in  order 
to  isolate  additional  factors  which  affect  group  performance.  By 
systematically  varying  factors  thought  to  affect  group  interaction,  the 
researcher  should  be  able  to  discover  which  component  of  the  model  each 
factor  affects.  The  components  could  then  be  further  partitioned  to  represent 
the  influence  of  the  new  factors.  In  this  way,  it  should  be  possible  to 
extend  the  model  to  address  a  variety  of  theoretical  questions  by  elaborating 
the  partition  appropriately.  For  example,  instructions  designed  to  modify  the 
interaction  of  group  members  could  be  employed  to  assess  the  effect  of  task- 
irrelevant  behavior  on  each  of  the  three  components  of  performance.  It  may  be 
possible  to  devise  similar  manipulations  to  examine  almost  any  aspect  of 
group  behavior  that  is  of  theoretical  interest. 


Footnotes 


^This  project  was  supported  by  OMR  Contract  N00014-80-C-0639.  Experiment 
1  was  previously  reported  in  more  detail  in  Casey,  Mehle,  &  Gettys  (Mote  1). 
He  illustrate  the  two-component  version  of  the  model  presented  in  the 
present  investigation  with  data  originally  presented  in  Casey,  et  al.  He 
would  like  to  thank  Richard  Reardon  for  comments  made  on  an  earlier  draft  of 
this  paper.  He  would  also  like  to  thank  Jason  Beckstead,  Pamela  Casey,  Carol 
Manning,  and  Peter  Engelmann  for  their  help  with  various  stages  of  this 
project . 

It  is  theoretically  possible  for  the  sum  of  the  two  solo  individual's 
information  stores  to  be  less  than  that  of  the  most  knowledgeable  solo 
individual  if  one  or  both  solo  individuals  possess  misinformation.  It  is  also 
possible  for  the  information  possessed  by  one  solo  individual  to  be  a  proper 
subset  of  the  information  possessed  by  another.  He  will  neglect  these 
interesting,  but  somewhat  pathological  possibilities  for  the  present  (cf. 
Casey.  Mehle  &  Gettys,  Mote  1). 
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ATTN:  Dr.  Gary  Poock 


Human  Engineering  Branch 
Code  8231 

Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

ATTN:  Mr.  Warren  Lewis 


Scientific  Advieer 
Coenandant  of  the  Marine  Corps 
Code  RD-1 

Washington,  D.C.  20380 
ATTN:  Dr.  A.L.  Slafkosky 


CIHCLANT  FLT  HQS 
Code  04E1 

Norfolk,  VA  23511 


Dr.  Janes  McGrath 


Naval  Material  Conn and 
NAVMAT  0722  -  KM.  508 
800  North  Quincy  Street 
Arlington,  VA  72217 


Naval  Air  Systens  Connand 
Hunan  Factors  Progrsns 
NAVA1R  340F 
Washington,  D.C.  20361 


ATTN: 


ander 


Connand.  Control  &  Connun icat ions 
Code  D  lOOA,  Dcvelopnent  Center 
Marine  Corps  Developnent  6 
Education  Connand 
Quant ico,  VA  22134 

ATTN:  Mr.  Fred  M.  Griffee 


Navy  Personnel  Research 
and  Developnent  Center 
Connand  and  Support  Systens 
San  Diego,  CA  92152 

ATTN:  Dr.  Robert  Blanchard 


Hunan  Factors  Engineering 
Division 

Naval  Air  Developnent  Center 
Waminster,  PA  18974 

ATTN:  CDR  J.  Funaro 


Hunan  Factors  Engineering  Division 
Naval  Air  Developnent 
Waminster,  PA  18974 

ATTN:  Dr.  Julie  Hopson 


Naval  Medical  R  4  D  Connand 
Code  44 

Naval  Medical  Center 
Bethesda,  MD  20014 

ATTN:  CDR  Robert  Biersner 


Hunan  Factors  Branch 
Code  3152 

Naval  Weapons  Center 
China  Lake,  CA  93555 

ATTN:  Mr.  Jeffrey  Grossnan 


Behavioral  Sciences  Departnent 
Naval  Medical  Research  Institute 
Bethesda,  MD  20014 

ATTN:  Dr.  Arthur  Bachrach 


Hunan  Factors  Engineering  Branch 
Code  1226.  „ 

Pacific  Missile  Test  Center 
Point  Mugu,  CA  93042 


Naval  Health  Research  Center 
San  Diego,  CA  92152 

ATTN:  CDR  Thonas  Berghage 


Naval  Postgraduate  School 
Code  55MP 

Monterey,  CA  93940 
ATTN:  CDR  C.  Hutchins 


Hunan  Factors  Engineering  Branch 
Subnarine  Medical  Research  Lab 
Naval  Subnarine  Base 
Groton,  CT  0340 

ATTN:  Dr.  George  Moeller 


Head 

Aerospace  Psychology  Departnent 
Code  L5 

Naval  Aerospace  Medical  Research  Lab 
Pensacola,  FL  3250$ 


Technical  Director 
U.S.  Amy  Research  Institute 
5001  Eisenhower  Avenue 
Alexandra,  VA  22333 


Director,  Organisations  and 
Systens  Research  Laboratory 
U.S.  Amy  Research  Institute 
5002  Eisenhower  Avenue 
Alexandria,  VA  22333 


U.S.  Air  Force  Office  of 
Scientific  Research 
Life  Sciences  Directorate,  ML 
Bolling  Air  Force  Base 
Washington,  D.C.  20332 


Chief,  Systems  Engineering  Branch 
Hunan  Engineering  Division 
DSAF  AMRL/HES 

Wright -Patter son  AFB,  OH  45433 


Chief  Scientist 
AFHRL/CCM 

Brooks  AFB,  TZ  78235 
ATTM:  Dr.  Earl  Alluisi 


Admiralty  Marine  Technology 
Establishment 
Applied  Psychology  Unit 
Tcadington,  Meddle sex  TW11  OLM 
EMGLAMD 

ATTN:  Dr.  Kenneth  Gardner 


Director.  Huaan  Factors  Wins 
Defense  a  Civil  Instftyte  ol 
Envirowtatal  Medicine 
Post  Office  Box  2000  .  .  , 
Downsviev,  Ontario  M3M  3B9 


Institute  for  Defense  Analyses 
400  Army-Mayy  Drive 
Arlington,  VA  22202 

ATTM:  Dr.  Jesse  Orlansky 


Massachusetts  Institute  of  Technology 
Department  of  Mechanical  Engineering 
Caaridge,  MA  02139 

ATTM:  DR.  T.B.  Sheridan 


Decision  Research 
1201  Oak  Street 
Eugene,  OR  97401 

ATTM:  Dr.  Paul  Slovic 


Virginia  Polytechnic  Institute 
and  State  University 
Department  of  Industrial  Engineering 
Blacksburg,  VA  24061 

ATTM:  Dr.  Harry  Snyder 


Stanford  University 
Department  of  Pave no log] 
Stanford,  CA  94305 

ATTM:  Dr.  Amos  Tversky 


Director,  Applied  Psychology  Unit 
Medical  Research  Council 
15  Chaucer  Road 
Cambridge,  CB2  2EF 
EMGLAMD 

ATTM:  Dr.  A.D.  Baddeley 


Defense  Technical  Information  Center 
Cameron  8tation,  Bldg.  5 

tlexandria,  VA  22314 
12  eye) 


Defense  Advanced  Research  Projects 
System  Sciences  Office 
1400  Wilson  Blvd. 

Arlington,  VA  22209 

ATTM:  Dr.  Judith  Daly 


MAS-Mational  Research  Council 
2101  Constitution  Ave..  M.W. 
Washington,  D.C.  20418 

ATTM:  Dr.  Robert  T.  Hennessy 

Perceptronics,  Inc. 

6271  Variel  Avenue 
Woodland  Hills,  CA  91364 

ATTM:  Dr.  M.G.  Samet 


Virginia  Polytechnic  Institute 
and  8tate  University 
Human  Factors  Laboratory 
130  Whittemore  Hall 
Blacksburg,  VA  24061 

ATTM:  Dr.  Robert  Williges 


Human  Factors  Research,  Inc. 
5775  Dawson  Avenue 
Goleta,  CA  93017 

ATTM:  Dr.  Robert  R.  Mackie 


University  of  Southern  California 
Director,  8oeial  Science  Research 
Institute 

Los  Angeles,  CA  90007 
ATTM:  Dr.  Hard  Edwards 


University  of  Coloredo 
Institute  of  Behavioral  8cience 
Soon  201 

Boulder,  CO  80309 
ATTM:  Dr.  Kenneth  Baesond 


Catholic  University 
Departaent  of  Psychology 
Washington,  D.C.  2006* 

ATTM:  Dr.  Janes  H.  Howard,  Jr. 


Rice  University 
Departaent  of  Psychology 
Houston,  TZ  77001 

ATTM:  Dr.  Hilliaa  Howell 


University  of  Illinois 
Departaent  of  Psychology 
Urbane,  IL  61801 

ATTM:  Dr.  Christopher  Hickens 


Decision  Research 
1201  Oak  8treet 
Eugene,  OR  97601 

ATTM:  Dr.  Baruch  Fischhoff 


University  of  Virginia 
School  q£  Engineering  and 
Applied  8cience 
Charlottesville,  VA  22901 

ATTM:  Dr.  Andrew  P.  Sage 


University  of  Wisconsin 
Depertaent  of  Psychology 
Madison,  WI  53706 

ATTM:  Dr.  Lola  Lopes 


Alphatech.  Inc. 

3  New  England  Industrial  Park 
Burlington,  MA  01803 

ATTM:  Mr.  Joseph  G.  Wohl 


Decision  Science  Consort iua 
Suite  721 

EH.1 SSSSPlm* 

ATTM:  Dr.  Rex  Brown 


Bolt  Beranek  A  Mewaan,  Inc. 
Inforaation  Sciences  Division 
50  Moulton  Street 
Csabridge,  MA  02238 

ATTM:  Dr.  Richard  W.  Pew 


Analytics,  Inc. 
2500  Maryland  Road 
Willow  Grove,  PA 


19090 


ATTN:  Dr.  Wayne  Zachary 


University  of  Chicago 
Graduate  School  of  Business 
1101  E.  58th  8treet 
Chicago,  IL  60637 

ATTM:  Dr.  Hillel  Einhorn 


Decision  Science  Consort iua 
Suite  721 

7700  Leesburg  Pike 
Falls  Church,  VA  22043 

ATTM:  Dr.  Marvin  Cohen 


Duke  University 
Graduate  8chool  of  Business 
Adainistration 
Durhaa,  MC  27706 

ATTM:  Dr.  John  Payne 


